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1 . INTRODUCTION 


The  classical  approach  to  sonar  target  detection  has 
been  to  provide  a sonar  operator  with  visual  displays  of  the 
information  received  by  the  sonar  system.  This  approach  is 
known  to  have  deficiencies  which  have  motivated  the  investiga- 
tion of  the  use  of  a computer  to  aid  in  the  detection  process. 

A computer-  ai degL-dftt.ee t ion  model  was  developed#  under 

" ' * """""i"  1 "■■—.»  i M,ii  — 

Contract  NObsr-93352.  ^his  model  is  currently  being  validated 
under  the  subject  contract  by  the  application  of  the  model  to 
a large  volume  of  recorded  sea  test  data.  A detailed  description 
of  the  data  base  and  the  data  processing  procedures  to  be  used 
is  contained  in  the  first  quarterly  report  under  the  subject 
contract  date. 

Hie  results  obtained  by  processing  twelve  sea  test 
runs  cure  presented  in  this  second  quarterly  report. hfA 
derivation  of  the  transformation  from  peak  height  to\log  likelihood 
ratio  is  also  included  in  Appendix  A. 
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2.  RESULTS  OBTAINED  FROM  TWELVE  SEA  TEST  RUNS 


(U)  Twelve  sea  test  runs  recorded  during  TECHEVAL  sea 

trip  two  have  been  processed  as  described  in  the  first  quarterly 
report.  The  resulting  false  alarm  rate  curves  are  presented 
in  Section  2.1  below.  The  results  obtained  by  applying  the 
computer- aided  detection  model  to  target  tracks  are  presented 
in  Section  2.2  below. 

2.1  False  Alarm  Rates 


(U) 


To  provide  a basis  for  comparison  ofjGAD  effects  on 
reverberation  a large  set  of  noise  was  generated  and  passed 

C. 

through  the  CAD  process.  Initially  4.2  x 10  independent 
samples  of  Gauss  noise  were  obtained.  This  sequence  was 
considered  to  be  a sampled  time  sequence  with  a sampling  rate  of 
1,000  samples  per  second.  The  sampled  time  sequence  was  band- 
limited  by  the  application  of  a digital  5th  order  Butterworth 
filter  with  3 dB  points  at  100  and  200  Hz.  The  bandlimited 
noise  was  passed  through  a linear  rectifier  followed  by  a 10 
sample  finite  time  perfect  averager.  The  resulting  "envelope" 
sequence  was  divided  into  396  sections  each  containing  10752 
samples.  Hie  data  was  first  thresholded  and  sorted  to  obtain 
the  curve  of  false  alarm  rate  vs  log  likelihood  ratio  shown  in 
Fig.  1.  Secondly,  the  data  was  processed  through  the  CAD  model 
and  sorted  to  obtain  the  curve  shown  in  Fig.  2.  Each  of  these 
curves  can  be  accurately  represented  by  an  expression  of  the 
form; 


FAR  = 


u> 


\ 


I 

I 

: 


where 

FAR 

L 

A,  B,  and  C 


False  alarm  rate,  SEC”^, 
Log  likelihood  ratio,  and 
are  constants. 
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The  values  of  A,  B,  and  C shown  in  Table  I were  determined  by 
a least  mean  square  £>i^  process.  The  results  obtained  by 
evaluating  Eq.  (1)  are  plotted  on  Figs.  1 and  2. 


(C) 


WITHOUT  CAD 

WITH  CAD 

A 

1.9276 

2.0273 

. 

B 

- .5560 

- .6623 

C 

- .0317 

- .0065 

TABLE  I 

FALSE  ALARM  RATE  COEFFICIENTS  FOR  NOISE 


| 


(U)  From  the  curves  shown  in  Figs.  1 and  2 it  is  clear 

that  the  CAD  model  has  adverse  effects  on  the  noise.  This 
result  is  not  unexpected  and  is  more  than  compensated  for 
by  the  benefits  derived  from  tracking. 

(C)  .As  described  in  the  first  quarterly  report  it  is 

necessary  to  separate  signal  information  from  noise  information 
in  processing  the  recorded  sea  data  to  obtain  the  desired 
results.  The  procedures  used  for  accomplishing  this  separation 
have  been  expanded  from  those  described  in  the  first  quarterly 
report  Due  to  the  sampling  techniques  used  in  digitizing  the 
TECHEVAL  data  the  submarine  track  usually  occurs  about  1.5 
seconds  from  the  beginning  of  the  cursor  gate.  Often  multiple 
paths  to  the  target  exist  causing  multiple  parallel  tracks. 

These  multiple  tracks  are  usually  confined  to  the  first  five 
seconds  of  the  cursor  gate.  For  this  reason  no  attempt  is  made 
to  use  information  in  the  first  five  seconds  of  the  cursor  gate 
to  develop  false  alarm  rate  curves.  This  procedure  successfully 
eliminates  most  target  tracks.  As  a further  precaution,  in 
processing  the  data  for  false  alarm  rate  curves  any  track  which 
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Figure  1.  Logarithm  of  the  False  Alarm 
Rate  vs  Logarithm  of  the  Likelihood 
Ratio  for  Noise  without  CAD  Processing 
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integrates  to  a log  likelihood  ratio  of  fourteen  is  called  as 
a target  track  and  is  not  counted  in  the  false  alarm  statistics. 

Using  data  from  twelve  sea  test  runs  the  false  alarm 
rate  curves  with  and  without  CAD,  shown  in  Figs.  3 and  4,  were 
generated.  Hie  false  alarm  rate  coefficients  are  given  in 
Table  II.  By  comparing  the  curves  of  Figs.  3 and  4 to  the 
base  curves  (derived  from  noise)  in  Figs'.  1 and  2 it  can  be 
seen  that  the  false  alarm  rates  are  slightly  higher  with  the 
sea  data  them  with  noise.  Due  to  the  quantity  of  data  processed 
at  the  present  time  the  quality  (or  confidence)  of  the  curves 
derived  from  sea  data  is  not  as  high  as  for  those  derived 
from  noise.  Hiis  condition  will  improve  as  more  data  is 
processed. 


WITHOUT  CAD 

WITH  CAD 

A 

.8616 

.9148 

B 

- .2675 

- .3342 

C 

- .0345 

< 

- .0208 

TABLE  II 

FALSE  ALARM  RATE  COEFFICIENTS  FOR  SEA  DATA 


2.2  Absolute  Performance  Measure 

Hie  performance  of  the  CAD  model  on  target  tracks  is 
condensed  to  a four  entry  track  information  package.  One 
track  information  package  is  generated  for  each  track  sequence 
processed.  Hie  four  entries  in  the  track  information  package 

are, 

(1)  N,  the  number  of  pings  which  were  processed 
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(2)  LLR,  the  maximum  track  log  likelihood  ratio 

obtained  by  the  CAD  model, 

(3)  S/N,  the  estimated  value  of  signal- to- noise 

for  the  N ping  sequence  going  into  the  CAD 
model , 

(4)  S/N; MAX,  the  maximum  signal-to-noise  ratio  which 

occurred  in  the  N ping  sequence  going  into 
the  CAD  model.* 

(U)  By  combioing  the  information  about  CAD  performance 

on  target  tracks  with  the  false  alarm  rate  information 
described  in  Section  2.1  it  is  possible  to  obtain  an  absolute 
measure  of  the  total  model  performance.  Several  means  are 
available  for  displaying  this  information.  One  technique 
described  in  the  first  quarterly  report  is  to  generate  modified 
ROC  curves.  This  technique  requires  a parameterization  of  the 
input  signal-to-noise  ratio.  Since  we  have  no  direct  controx 
over  the  input  signal-to-noise  ratio  the  required  parameterization 
must  be  accomplished  in  a somewhat  arbitrary  way  as  described 
in  the  first  quarterly  report.  To  avoid  this  arbitrary 
parameterization  a different  technique  for  displaying  the 
results  has  been  developed. 

(C)  For  each  track  information  package  the  maximum  track 

log- likelihood  ratio  can  be  used  to  index  the  false  alarm 
rate  curve  associated  with  the  output  of  the  CAD  model  to 
determine  the  false  alarm  rate  which  would  be  associated  with  a 
threshold  set  so  that  the  track  sequence  would  be  detected.  Using 
this  false  alarm  rate  and  the  estimated  value  of  signal-to-noise 
ratio  a point  can  be  plotted  on  a grid  of  signal-to-noise  ratio 
vs  false  alarm  rate.  By  repeating  this  process  for  all  available 
track  information  packages  with  a common  value  of  N a distribution 
of  points  on  the  grid  can  be  obtained.  Finally  the  points  on  the 

*lfcis  fourth  quantity  has  been  added  since  the  writing  of  the 
first  quarterly  report. 

jj 

CONFIDENTIAL 

jj 

— — ; — ■'  "T  — - "V,  ", '.'r*""—  i U.1.-.-  ■ ' . -M - 1 " ** 


grid  may  be  fitted  with  a quadratic  expression  to  obtain  an 
estimate  of  input  signal- to- noise  ratio  vs  false  alarm  rate  for 
each  N ping  sequence  processed  by  the  CAD  model.  This  process 
has  been  carried  out  and  the  results  are  shown  in  Figs.  5 through 
12  where  the  number  of  pings  is  varied  par ameterically . The 
signal-to-noise  ratios  are  given  in  dB  on  the  vertical  scale 
and  the  logarithm  (to  the  base  10)  of  the  false  alarm  rate  (in 
Sec~^)  is  given  on  the  horizontal  scale.  The  symbol  "X"  is  used 
to  represent  points  derived  directly  from  the  track  information 

packages  and  the  symbol  is  used  to  plot  the  curve  fit  function. 

Many  large  signal-to-noise  ratio  tracks  yield  points  which  are 
off  the  scale  of  the  graphs  shown  in  Figs.  5 through  12.  Even 
though  these  points  cure  not  shown  they  are  included  in  determin- 
ing the  coefficients  of  the  curve  fit  since  they  do  present 
valid  information  about  trends. 

(C)  To  provide  some  insight  into  the  benefits  derived 

from  tracking  it  is  interesting  to  determine  what  the  performance 
would  have  been  had  the  CAD  model  not  been  used.  An  alternate 
detection  procedure  would  be  to  use  the  largest  signal  occurring 
in  a track  for  detection  with  no  attempt  made  to  track.  If 
this  were  done  the  appropriate  false  alarm  rate  curve  would  be 
the  curve  given  in  Fig.  3 without  the  CAD  model.  It  is  important 
to  note  that  when  several  observations  are  allowed  (such  as  6) 
the  expected  value  of  the  largest  signal  may  be  considerably 
larger  than  the  average  signal  level.  To  obtain  a display  of 
the  performance  associated  with  this  type  of  processing  the 
maximum  signal-to-noise  ratio  which  occurred  was  transformed 
into  a log  likelihood  ratio  and  used  to  index  the  false  alarm 
rate  curve  without  CAD  to  determine  a false  alarm  rate.  The 
resulting  (FAR,  S/N)  pairs  were  processed  as  described  above 


to  obtain  the  results  shown  in  Figs.  13  through  20.  By  comparing 
corresponding  results  for  the  use  of  CAD  for  detection  vs  the 
use  of  the  peak  for  detection  two  conclusions  can  be  drawn: 
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S/N  VS  FALSE  ALAHrt  HATE  USING  CAD  FOH  DETECTION* 


FIGURE 


FIGURE  8 


VS  FALSh  ALrtrfM  KATE  USING  CAD  FOR  DETECT  1 0 N 


FIGURE  9 


FIGURE  10 


FIGURE  II 


LSt  ALAWM  K a Tbi  USING  CAP  FOK  DETECT  ION 


FIGURE  12 


FIGURE  13 


Lhk*  ha  fK  USING  PtiAK  FOP  DETECTION* 


S/N  VS  KALSt  aLAK'l  rtAffc.  USIN<j  P|£AK  FOM  DETECTION*  M=  3*  A= •‘I2»R69254*  fJ= 2*9461 07* 


FIGURE  15 


S/N  VS  *Lak.H  US  IMG  PtiAK  FOR  DETECT  ION » M=  5 , 


FIGURE  18 


OS I NO  peak  for  DETECTION 


FIGURE  19 


FIGURE  20 
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(C)  (1)  In  an  average  sense  the  detection  performance 

obtained  with  the  CAD  model  is  about  1 to  2 dfi  better  than 
detection  performance  obtained  using  the  peak  signal  for 
detection  in  the  low  false  alarm  rate  region. 

(C)  (2)  The  detection  performance  obtained  with  the 

CAD  model  is  much  more  consistent  then  that  obtained  using  the 
peak  signal  for  detection. 


\ 


it 


CONFIDENTIAL 


4 


17^7^71  UNCLASSIFIED 


3.  CONTINUING  WORK 

During  the  next  repcr  ting  period  additional  sea  test 
data  will  be  processed  to  increase  the  confidence  of  the  results  • 
presented  here.  Also  the  processing  of  the  operator  response 
information  will  begin. 


\ 
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APPENDIX  A.  DERIVATION  OF  THE  TRANSFORMATION 
PEAK  HEIGHT  TO  LOG  LIKELIHOOD  RATIO 

The  transformation  from  processor  output  peak 
amplitude  to  logarithm  of  likelihood  ratio  is  the  initial 
processing  stage  of  the  target  tracking  model.  Model  per- 
formance is  directly  related  to  the  accuracy  of  the  trans- 
formation. Ihe  validity  of  the  log  likelihood  conversion 
algorithm  is  in  turn  dependent  upon  an  accurate  statistical 
description  of  processor  output  amplitude  in  a multi-ping  signal- 
plus-noise  environment.  The  purpose  of  this  Appendix  is  to  present 
two  different  statistical  approximations  to  a multi-ping  signal- 
plus-noise  distribution  and  to  document  thei^  associated 
likelihood  ratio  transformations. 

The  likelihood  ratio,  Lx,  associated  with  a processor 
output  peak  of  amplitude  x is  given  by 


P U) 
*2* 


u) 


where  Pg  is  the  probability  density  function  of  amplitude  in 
a processor  output  exhibiting  a signal- to- noise  ratio  S and  P 

n 

is  the  probability  density  function  of  amplitude  in  noise  alone 
processor  output. 

One  immediate  problem  arises  since  the  likelihood  ratio 
defined  above  is  dependent  upon  the  average  signal-to-noise 
ratio  S assumed  at  the  processor  output.  In  order  to  inplement 
the  likelihood  ratio  conversion,  the  value  of  S must  be  fixed  as 
a system  parameter.  On  the  other  hand,  the  actual  value  of  S 
cannot  be  known  before  the  fact  and  it  may  vary  widely  from  one 
contact  to  another.  One  solution  to  this  problem  is  to  set  S 
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in  the  range  of  the  minimum  detectable  signal- to- noise  ratio  at 
the  output  of  the  processor.  When  this  is  done,  likelihood  ratio 
estimates  computed  according  to  Eq.  1 will  be  inexact  only  for 
signal-to-noise  ratios  somewhat  larger  or  smaller  than  S.  This 
inexactness  will  not  affect  overall  system  performance  due  to 
the  fact  that  the  smaller  signals  are  presumably  non-detectable 
at  the  processor  output  and  that  the  larger  signals  require 
less  processing  gain  for  detectability,  Thus,  when  S is  set  at 
a marginally  detectable  signal-to-noise  ratio,  the  tracking 
model  is  optimized  to  provide  maximum  processing  gain  for  those 
signals  that  are  presently  only  barely  detectable.  In  the 
validation  study,  the  region  of  minimal  detectable  signal-to- 
noise  ratio  at  the. output  of  the  correlator  is  from  10  dB  to 
12  dB. 

The  next  problem  in  deriving  a log  likelihood  trans- 
formation is  to  determine  the  probability  density  functions  P 

S 

and  Pn  required  for  the  computation  of  Eq.  1.  Two  calsses  of 
probability  density  functions  have  been  used  to  date  to  derive 
log  likelihood  ratio  conversion  algorithms.  The  first  class  of 
functions,  discussed  in  Section  2 of  this  Appendix,  arise  from 
the  assumption  that  the  ping-to-ping  signal-plus-noise  statistics 
of  correlator  output  amplitude  are  described  by  the  envelope 
distribution  of  an  ideal  signal  in  Gaussian  noise.  The  second 
class  of  functions,  discussed  in  Section  3 of  this  Appendix, 
arise  from  the  assumption  that  signal-plus-noise  and  noise 
alone  correlator  output  amplitude  statistics  are  both  described 
by  the  Rayleigh  distributions  with  different  standard  deviations. 
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2.  LOG  LIKELIHOOD  RATIO  OF  IDEAL  SIGNAL  PLUS 
GAUSSIAN  NOISE  ENVELOPE 


If  the  assumption  is  made  that  signal  amplitude  is 
constant  from  ping- to- ping,  then  echo  returns  following  the 
correlator  are  similar  to  short  CW  pulses  of  the  same  signal 
amplitude.  The  linear  rectifier  and  averager  following  the 
correlator  then  act  as  an  envelope  detector  for  these  pulses. 
Thus,  the  probability  density  functions  required  for 
evaluating  Eq.  1 are  those  which  describe  the  envelope  of  a 
sine  wave  plus  random  noise.  These  density  functions  are  given 
by  S.  0.  Rice*  is 


Pg(X)  - e"S/*  e^5  Iq(/2  /S75  ) 


p (x)  = — e ^ 
N e 


where  X is  the  height  of  the  envelope 

N is  the  average  noise  power 

S is  the  average  signal  power 

I is  the  modified  Bessel  function  of  order  o 
o 

Using  these  two  equations  to  compute  likelihood  ratio, 
Bq.  1 yields 


L*  = e"S/1*  Iq(/5  /S7N 


*Rice,  S.  0.,  H Mathematical  Analysis  of  Random  Noise,"  Bell 
System  Technical  Journal,  Vol.  24,  1945,  p.  100. 
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As  given  by  Rice,  the  output  noise  mean,  (i,  is  /Ntt/2  and  the 
output  noise  standard  deviation  a is  /N(2-ti/2)  . If  r denotes 
the  envelope  height  in  units  of  a relative  to  u,  then 

r - (X-u)/o 


* A(  2-tt/2)  • r + J^72  . 

Substitution  for  X in  the  equation  for  above  yields 


I*  » e"S/K  I0[/§7S  ‘ r + ft  ] (2) 

The  conversion  algorithm  for  correlator  output  amplitude 
samples  X is  then  obtained  by  taking  the  natural  logarithm  of 
Eq.  2 where  X has  been  converted  to  r using  measured  values  of 
u and  a.  This  has  been  done  in  Fig.  A-l  where  the  logarithm 
of  Eq.  2 has  been  plotted  as  a function  of  r for  three  values 
of  S/N  that  yield  average  output  signal-to-noise  ratios  of  10  dB, 
11  dB  and  12  dB. 

The  curves  that  appear  in  Fig.  A-l  thus  yield  the  fruitful, 
result  that  the  logarithm  of  Eq.  2 is  closely  approximately  by 
a straight  line  in  the  region  of  interest  (i.e.  values  of  r 
from  2 to  6 which  correspond  to  output  signal- to-noise  ratios 
from  6 dB  to  15.5  dB) . The  logarithm  of  likelihood  ratio 
conversion  of  a measured  output  peak  of  amplitude  X then  reduces 
to 
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in  Ljj  - a • r + b 


+ b 


tnLv=-*X+b-— 

a a o 


• » 


I 

(3) 


where  a and  b are  the  slope  and  slope  intercept  values  defining 
the  appropriate  straight  line  in  Fig.  A-l  and  u and  a are 
measured  values  of  correlator  output  noise  mean  and  standard 
deviation.  For  example,  if  the  expected  correlator  output 
signal- to- noise  ratio  is  to  be  set  at  10  dB,  then  Fig.  A-l  is 
examined  and  values  of  a = 2.0  and  b = -2.86  are  computed  as 
(the  values)  determining  the  straight  line  marked  10  dB.  In 
this  case  then,  the  transformation  from  peak  height  X to 
logarithm  of  likelihood  ratio  In  is  given  by 


4n  * 2.0  £ - 2.86 


2.0u 

CT 


In  summary,  the  assumption  that  correlator  output 
peak  statistics  are  described  on  a multiping  basis  by  the 
distribution  of  constant  amplitude  ideal  signal  plus  Gaussian 
noise  yields  the  easily  implemented  linear  transformation  in 
Eq.  3 from  peak  height  to  logarithm  of  likelihood  ratio. 
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3.  LOG  LIKELIHOOD  RATIO  OF  RAYLEIGH  SIGNAL 
PLUS  RAYLEIGH  NOISE 

(U)  Although  the  log  likelihood  ratio  conversion  discussed 

in  Section  2 is  simple  to  implement,  the  assumptions  upon  which 
it  is  based  seem  rather  stringent.  In  particular,  the  assumption 
that  consecutive  echo  cycles  contain  ideal  constant  amplitude 
signals  plus  Gaussian  noise  is  not  confirmed  by  empirical  data 
which  indicates  that  peak  heights  have  a larger  variation  than 
that  expected  in  an  ideal  case.  For  this  reason,  a "noisy" 
signal  distribution  was  considered.  Ibis  distribution  is  to 
be  denoted  as  Rayleigh  signal  plus  Rayleigh  noise. 

(U)  The  Rayleigh  probability  density  function,  which  is 

identical  to  the  Rice  noise  alone  distribution,  is  given  by 


The  signal-plus- noise  hypothesis  investigated  in  this  section 
assumes  that  both  signal-plus-noise  and  noise  alone  peaks  are 
distributed  by  Rayleigh  density  functions  Pg  and  PN  with 
different  parameters  as  and  a^.  Hence 


PgU) 


2a: 


and 


PU(S) 


2a, 


n 


Under  the  assumption  of  the  above  density  factors,  the 
likelihood  ratio  measurement  is  given  by 
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Hie  mean  values  u8  and  un  and  standard  deviation  values  ag 
and  on  of  the  signal  and  noise  distributions  cure  given  by 


Ug  * as  /tt/2 


un  = an  szn 


ag  * ag  /2  - tt/2 


an  * an  /2  - ^2 


Consequently,  amplitude  values  X,  when  measured  in  units  r of 

noise  standard  deviation  o relative  to  the  noise  mean  u„  are 

n n 

given  by 


X = on  r + un 


= an  / 2 - tt/2  r + an  /tt/2 

Substituting  the  above  for  X in  the  expression  above  for  L^, 
the  following  is  obtained 

2 an2fklr  + k2^T~’S  " 

a_  2a  2a 

r . — « n s 

^ “ ” e 

as 


tV  + k2]2(i 


d2 

2a,2> 
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where  kA  « /2  - tt/2  and  k2  = /tt/2  . 

Finally,  taking  log  likelihood  ration,  we  obtain 


tn  Ljj*  tn(-^)  + (*j  - -2-^)  [kjr  + k212 
as  2as 


The  relation  between  — and  correlator  output  signal- to- noise 
ratio  is  easily  obtainid.  For  example,  since  the  tracking - 
model  is  to  be  optimized  for  10  dB,  it  is  required  that 


u -u 

10  = 20  log  (~s  n) 
an 


a k0-a„  k_ 
20  log  (•?  2 Vn  2) 


r tts  k2  k2v 


20  1o^  (^r  i r - ir> 

an  K1  K1 


Consequently, 


a k.  a 

rr  m /IS  + 1 = 2.65  and  — = .377 

an  k2  as 


Ihus,  to  optimize  Eq.  4 for  a 10  dB  correlator  output  signal- 
to- noise  ratio,  the  value  .377  is  substituted  in  Eq.  4 for 
an 

— and  the  •following • conversion  algorithm  results. 
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tin  1^  » -ln( . 377)  2 + (-|  - .3772)  [kjT  + k2]2 
= -1.275  + (0.706)  r + (0.185)r2 

a. 

(U)  Equation  4 has  been  evaluated  for  values  of  — 

as 

corresponding  to  correlator  output  signal-to-noise  ratios 
of  10  dB  and  11.5  dB.  The  resulting  curves  in  which  -in 
is  plotted  as  a function  of  r appear  in  Pig.  A- 2.  The  quadratic 
transformation  indicated  in  Eq.  4 may  be  simplified  further  by 
computing  a least  means  square  linear  fit  in  the  region  of 
interest  to  the  curves  in  Fig.  A- 2.  This  has  been  done  for  the 
10  dB  curve  and  the  resulting  linear  fit  is  indicated  by  the 
dotted  line.  , The  slope  and  slope  intercept  values  a and  b 
yield  a simplified  log  likelihood  transformation  given  by 


In  L x = 2.45r  - 5.2 

(U)  In  summary,  the  assumption  of  a Rayleigh  signal  plus 

Rayleigh  noise  distribution  yields  a quadratic  transformation 
from  peak  height  X to  log  likelihood  ratio  that  is  quite 
practical  to  implement  and  which  further  may  be  approximated 
by  a linear  transformation  in  the  range  of  interesting  signal-to- 
noise  ratio.  ttiis  form  of  log  likelihood  ratio  transform  is 
being  used  in  the  validation  study. 
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